1. Field of the Invention
The present invention generally relates to the field of computer memory management systems, and more particularly relates to a method and system for managing pinned kernel memory used as data buffers for time sensitive processing.
2. Description of Related Art
Many types of data processing equipment utilize virtual memory that allows data being handled by the processor to be stored in a combination of physical memory and mass storage. Virtual memory allows a processor to utilize memory in excess of that available in actual physical memory connected to the processor. The operation of virtual memory results in some data being stored in a mass storage device, such as a hard disk drive, and the swapping, or paging, of data between physical memory and mass storage as data stored in mass storage is required to be accessed by the processor. An attempt to access data that is stored in mass storage, which is referred to as a page fault, requires that the data be swapped into physical memory. The swapping of data between physical memory and mass storage may introduce a relatively long delay in access to the data as compared to accessing data already stored in physical memory. An excessive amount of data swapping from physical memory to mass storage is referred to as “thrashing” and has a detrimental impact upon system performance.
Electronic data processing equipment are often required to perform data communications with remote equipment. The data produced by a data storage system or other data processing systems must often be relayed to other computers or to other data processing equipment in order to be useful. Efficient utilization of processing resources to perform these data communications tasks is required in order to minimize equipment cost and maximize the usefulness of the data processing equipment.
The availability of powerful microprocessors has also made the distribution of processing among several computers, which are interconnected together by a communications network, an attractive alternative to using a central, monolithic computer system. A series of computers which are interconnected by a network may also host computer applications that are able to partition their processing among several nodes in order to realize greater performance. Such computers are required to implement data communications processing which efficiently utilize computing resources.
Interconnected computer systems consist of a number of separate computing systems coupled with an inter-processor communication mechanism, such as a network or a communications switch. Each computing system has its own processor, memory, and I/O subsystems and each typically runs a separate instance of an operating system. Interconnected computer systems may also be configured to allow applications to run in an abstract fashion independently of a given computing system, a manner which treats all nodes in a cluster as equivalent. These systems may also allow a computer system to access data processing resources that are located in any other computer system that is part of the interconnected computer system.
Different approaches are used to provide access to data processing resources on one computer system to all of the computer systems that are part of an interconnected computer system. One application of this architecture provides one or more centrally accessible remote data storage devices that communicate to all interconnected processors. Distributed file systems, such as NFS, AFS, and DFS, are all operable in a manner which allows applications on one processor to access remote data storage devices. These file systems often present abstract data storage functions to the application and preclude the application from controlling the storage details of the data storage.
In order to maximize processing efficiency and meet processing time requirements, the data communications processing in a computer system, or other data processing equipment that is part of an interconnected computer system, often stores data in temporary memory buffers that are held in a fixed physical memory location within the data processing equipment. The data buffers are stored in the memory space used by the operating system of the processing equipment, which is referred to herein as the “kernel” memory. The kernel memory used for the data communications buffer is configured so that the data in that memory is not paged out of real memory so that the data is ready for immediate access by the processing hardware. Configuring the memory to prevent paging out of the data stored in that portion of memory is referred to as “pinning” the memory. This ensures that page faults do not occur for data access to the temporary buffer. Pinning kernel memory has an adverse impact upon other processing operations within the data processing equipment since the kernel memory that is pinned by one process is not available to support processing of other processing tasks. The pinning of kernel memory also limits the memory management flexibility of the processing module that is pinning the memory.
Prior art data communications processing components of data processing equipment pin a fixed amount of kernel memory in a large, contiguous memory block and stores communication data into buffers that are allocated within that fixed kernel memory block. This operation requires that this fixed amount of kernel memory be pinned and allocated to processing by the external data communications processing component during initialization. That fixed amount of contiguous kernel memory then remains pinned during all subsequent operations. Selection of the amount of fixed kernel memory to allocate to external data communications processing requires trading off external data communications performance with the processing performance of other processing tasks performed by the data processing equipment. Allocation of an insufficient amount of pinned kernel memory to external data communications processing results in queuing of external data communications data messages, which degrades the performance of the data processing equipment as observed by external users. Allocation of too much pinned kernel memory results in degraded performance of other processes within the data processing equipment since the amount of kernel memory available to those other processes is reduced. The identification of the proper amount of pinned kernel memory to allocate to the external data communications processing is hindered by the fact that the amount is dependent upon a peak external data communications demand.
Therefore a need exists to overcome the problems with the prior art as discussed above, and particularly for a way to more efficiently utilize system kernel memory within data processing equipment to support time sensitive processing tasks such as external data communications processing.